Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update docker-images to version 72 #14810

Merged
merged 2 commits into from
Nov 2, 2022
Merged

Update docker-images to version 72 #14810

merged 2 commits into from
Nov 2, 2022

Conversation

hashhar
Copy link
Member

@hashhar hashhar commented Oct 27, 2022

Release notes

(x) This is not user-visible or docs only and no release notes are required.

@hashhar hashhar added tests:all Run all tests tests:all-product Run all product tests labels Oct 27, 2022
@cla-bot cla-bot bot added the cla-signed label Oct 27, 2022
nineinchnick
nineinchnick reviewed Oct 27, 2022
View reviewed changes
pom.xml Outdated Show resolved Hide resolved
@hashhar hashhar force-pushed the hashhar/docker-images-update branch from f779e8f to b8ee680 Compare October 27, 2022 16:36
@nineinchnick
Copy link
Member

I ran some PTs using this branch on an M1 MacBook and they seem to work fine. For example, this one:

./testing/bin/ptl test run --environment singlenode-postgresql --config config-default --logs-dir logs/ --timeout 2h -- -g configured_features,postgresql

takes 1 minute 40 seconds and it can't even finish (timeouts) when run on master.

@hashhar
Copy link
Member Author

hashhar commented Oct 28, 2022

hdp3, suite-azure seems like legitimate failure (or flakiness).

spark related things fail because Spark container's healthchecks fail - https://github.com/trinodb/docker-images/pull/147/files#r1007802848.

@hashhar
Copy link
Member Author

hashhar commented Oct 28, 2022

Something weird with Spark - the env stays up and container remains healthy locally. Will see what happens when running a suite.

@hashhar
Copy link
Member Author

hashhar commented Oct 31, 2022

@nineinchnick found the issue - seems like a dualstack problem. Health-check is trying to hit IPv6 endpoint for some reason.

2022-10-31T15:16:17.8815931Z 2022-10-31T15:16:17.877Z	WARN	main	io.trino.tests.product.launcher.env.Environment	current health logs: [HealthStateLog(start=2022-10-31T15:16:08.037787725Z, end=2022-10-31T15:16:08.28324707Z, exitCode=7, output=  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2022-10-31T15:16:17.8820623Z                                  Dload  Upload   Total   Spent    Left  Speed
2022-10-31T15:16:17.8820944Z 
2022-10-31T15:16:17.8823722Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to ::1: Cannot assign requested address
2022-10-31T15:16:17.8824233Z )]
2022-10-31T15:16:17.8826166Z 2022-10-31T15:16:17.877Z	WARN	main	io.trino.tests.product.launcher.env.Environment	health logs: []
2022-10-31T15:16:17.8963733Z 2022-10-31T15:16:17.887Z	WARN	main	io.trino.tests.product.launcher.env.Environment	current health logs: InspectContainerResponse(args=[--master, local[*], --class, org.apache.spark.sql.hive.thriftserver.HiveThriftServer2, --name, Thrift JDBC/ODBC Server, --conf, spark.hive.server2.thrift.port=10213, spark-internal], config=ContainerConfig(attachStderr=false, attachStdin=false, attachStdout=false, cmd=[spark-submit, --master, local[*], --class, org.apache.spark.sql.hive.thriftserver.HiveThriftServer2, --name, Thrift JDBC/ODBC Server, --conf, spark.hive.server2.thrift.port=10213, spark-internal], domainName=, entrypoint=null, env=[HADOOP_USER_NAME=hive, PATH=/spark/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin, JAVA_HOME=/usr/lib/jvm/zulu-11, LANG=en_US.UTF-8, SPARK_HOME=/spark], exposedPorts=[10213/tcp], hostName=spark, image=ghcr.io/trinodb/testing/spark3-iceberg:70, labels={io.trino.git.hash=f8fd83bd732f667e65c95c78e4c6624ec17c9f94, io.trino.tests.product.launcher.env.Environment.ptl-started=true, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS, org.testcontainers=true, org.testcontainers.sessionId=41e0f514-8c50-4a70-8331-d6d40a315ba6}, macAddress=null, networkDisabled=null, onBuild=null, stdinOpen=false, portSpecs=null, stdInOnce=false, tty=false, user=, volumes=null, workingDir=/spark/jars, healthCheck=HealthCheck(interval=10000000000, timeout=5000000000, test=[CMD-SHELL, curl -f http://localhost:10213/], retries=null, startPeriod=10000000000)), created=2022-10-31T15:15:57.130567767Z, driver=overlay2, execDriver=null, hostConfig=HostConfig(binds=[], blkioWeight=0, blkioWeightDevice=null, blkioDeviceReadBps=null, blkioDeviceWriteBps=null, blkioDeviceReadIOps=null, blkioDeviceWriteIOps=null, memorySwappiness=null, nanoCPUs=0, capAdd=null, capDrop=null, containerIDFile=, cpuPeriod=0, cpuRealtimePeriod=0, cpuRealtimeRuntime=0, cpuShares=0, cpuQuota=0, cpusetCpus=, cpusetMems=, devices=null, deviceCgroupRules=null, deviceRequests=null, diskQuota=null, dns=null, dnsOptions=null, dnsSearch=null, extraHosts=[], groupAdd=null, ipcMode=shareable, cgroup=, links=[], logConfig=LogConfig(type=json-file, config={}), lxcConf=null, memory=0, memorySwap=0, memoryReservation=0, kernelMemory=0, networkMode=480358e9dc126d76bc2845ce6892024f6aa6487ec043969ba5c5defb87e0e366, oomKillDisable=true, init=null, autoRemove=true, oomScoreAdj=0, portBindings={10213/tcp=[Lcom.github.dockerjava.api.model.Ports$Binding;@493da830}, privileged=false, publishAllPorts=false, readonlyRootfs=false, restartPolicy=no, ulimits=[Ulimit(name=nofile, soft=65535, hard=65535), Ulimit(name=nproc, soft=8096, hard=8096)], cpuCount=0, cpuPercent=0, ioMaximumIOps=0, ioMaximumBandwidth=0, volumesFrom=[], mounts=null, pidMode=, isolation=null, securityOpts=null, storageOpt=null, cgroupParent=, volumeDriver=, shmSize=67108864, pidsLimit=null, runtime=runc, tmpFs=null, utSMode=, usernsMode=, sysctls=null, consoleSize=[0, 0], cgroupnsMode=host), hostnamePath=/var/lib/docker/containers/1643e919f14f3d14fc31fd6fa3ad09480b0d9f6f2d78ee946ac90e7ef46b2ddb/hostname, hostsPath=/var/lib/docker/containers/1643e919f14f3d14fc31fd6fa3ad09480b0d9f6f2d78ee946ac90e7ef46b2ddb/hosts, logPath=/var/lib/docker/containers/1643e919f14f3d14fc31fd6fa3ad09480b0d9f6f2d78ee946ac90e7ef46b2ddb/1643e919f14f3d14fc31fd6fa3ad09480b0d9f6f2d78ee946ac90e7ef46b2ddb-json.log, id=1643e919f14f3d14fc31fd6fa3ad09480b0d9f6f2d78ee946ac90e7ef46b2ddb, sizeRootFs=null, imageId=sha256:b8a30b0786b540410480b5cdac12352efc29859121f2f2e278892f3954094ec9, mountLabel=, name=/ptl-spark, restartCount=0, networkSettings=NetworkSettings(bridge=, sandboxId=f7a874f3bbccc5121b20e508ab9f820d8523deb24adf6dafb93a84a996e2ef9c, hairpinMode=false, linkLocalIPv6Address=, linkLocalIPv6PrefixLen=0, ports={10213/tcp=[Lcom.github.dockerjava.api.model.Ports$Binding;@45658133}, sandboxKey=/var/run/docker/netns/f7a874f3bbcc, secondaryIPAddresses=null, secondaryIPv6Addresses=null, endpointID=, gateway=, portMapping=null, globalIPv6Address=, globalIPv6PrefixLen=0, ipAddress=, ipPrefixLen=0, ipV6Gateway=, macAddress=, networks={ptl-network=ContainerNetwork(ipamConfig=null, links=[], aliases=[tc-v3G5HDei, spark, 1643e919f14f], networkID=480358e9dc126d76bc2845ce6892024f6aa6487ec043969ba5c5defb87e0e366, endpointId=9838547dae9d1c34001fa25a49372ee9ef45ae7332afc9ad3beffc414d7f4496, gateway=172.23.0.1, ipAddress=172.23.0.4, ipPrefixLen=16, ipV6Gateway=, globalIPv6Address=, globalIPv6PrefixLen=0, macAddress=02:42:ac:17:00:04)}), path=spark-submit, processLabel=, resolvConfPath=/var/lib/docker/containers/1643e919f14f3d14fc31fd6fa3ad09480b0d9f6f2d78ee946ac90e7ef46b2ddb/resolv.conf, execIds=null, state=InspectContainerResponse.ContainerState(status=running, running=true, paused=false, restarting=false, oomKilled=false, dead=false, pid=19547, exitCode=0, error=, startedAt=2022-10-31T15:15:58.036101679Z, finishedAt=0001-01-01T00:00:00Z, health=HealthState(status=starting, failingStreak=1, log=[HealthStateLog(start=2022-10-31T15:16:08.037787725Z, end=2022-10-31T15:16:08.28324707Z, exitCode=7, output=  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

@hashhar
Copy link
Member Author

hashhar commented Oct 31, 2022

I'll see if changing trinodb/docker-images@3b5181f#diff-31c91f5c3ac99713c666b4c0b00c01ad430ddb4ee8686e6b279910b6f09a3e5dR46 to use IPv4 explicitly helps.

@hashhar hashhar mentioned this pull request Nov 1, 2022
Merged
@hashhar
Include container healthcheck logs for unhealthy containers
b1aac9f 
The product tests fail the environment startup if all containers are not
healthy and log the name of the unhealthy container. Since the logs
don't include the healthcheck logs from Docker it's impossible to find
why the container appeared unhealthy. This change additionally logs the
healthcheck responses from the Docker daemon for unhealhty containers.
@hashhar hashhar force-pushed the hashhar/docker-images-update branch from e44cb58 to d0c6077 Compare November 1, 2022 09:44
@hashhar hashhar marked this pull request as ready for review November 1, 2022 09:44
@hashhar hashhar requested review from losipiuk and ebyhr November 1, 2022 09:44
@hashhar hashhar mentioned this pull request Nov 1, 2022
Closed
@hashhar hashhar changed the title Update docker-images to version 70 Update docker-images to version 71 Nov 1, 2022
@hashhar hashhar marked this pull request as draft November 1, 2022 11:07
@hashhar hashhar force-pushed the hashhar/docker-images-update branch 2 times, most recently from ff178b7 to 67f07d0 Compare November 1, 2022 12:37
@hashhar hashhar mentioned this pull request Nov 1, 2022
Merged
@hashhar
Update docker-images to version 72
3755b94 
This also updates all centos7-oj11 based images to move to centos7-oj17
because of changes in
trinodb/docker-images@d67aab3
which change the images to use the oj17 base.
@hashhar hashhar force-pushed the hashhar/docker-images-update branch from 67f07d0 to 3755b94 Compare November 2, 2022 07:39
@hashhar hashhar marked this pull request as ready for review November 2, 2022 11:38
@hashhar
Copy link
Member Author

hashhar commented Nov 2, 2022

This is ready to review now. @ebyhr @nineinchnick @losipiuk

@hashhar hashhar changed the title Update docker-images to version 71 Update docker-images to version 72 Nov 2, 2022
@nineinchnick
Copy link
Member

@ebyhr CI is green, can we merge this? I need this to work one some product tests :-)

@hashhar hashhar merged commit 67b761c into master Nov 2, 2022
@hashhar hashhar deleted the hashhar/docker-images-update branch November 2, 2022 13:54
@github-actions github-actions bot added this to the 402 milestone Nov 2, 2022
@colebow colebow mentioned this pull request Nov 2, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ebyhr ebyhr ebyhr approved these changes

@nineinchnick nineinchnick nineinchnick approved these changes

@losipiuk losipiuk Awaiting requested review from losipiuk

Assignees
No one assigned
Labels
cla-signed tests:all Run all tests tests:all-product Run all product tests
Milestone
402
Development

Successfully merging this pull request may close these issues.
3 participants
@hashhar @nineinchnick @ebyhr